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(57) Abstract 

The system which performs the real-time rendering of HD video data containing the 
effect which the effect added to high definition (HD) video data added which added 
and real-time beforehand visualized . The system into which high definition 
television (HDTV) resolution video is edited using a computer contains the high 
definition video system which has connected at least the reference standard to the 
video system, and a high definition storage system. At least in a reference standard, 
resizer reformats high definition video data into resolution for real-time processing 
and prior visualization. 



Claim(s) 

Claim 1 It is a nonlinear video edit system. Store video data per data file. They are 
that random access computer reading is possible and the storage which can be re- 
written in. The storage which defines the image which has the scanning line with 
more said video data than 525, and has a component rate higher than 30 frames per 
second, It is the nonlinear editor which defines video program as a sequence of the 
part of said data file. The nonlinear editor which defines each part of a data file by 
the range in the reference to said data file, and said data file, The means which 
reads said video data from said storage according to said defined video program, 
Resizer which gives the output video data which has reception and the space 
resolution of the scanning line fewer than 525 for said video data read from said 
storage, and has a component rate lower than 30 frames per second, Nonlinear 
video edit system equipped with the display which has connected with the output of 
said resizer and displays said resized video data. 

Claim 2 The system according to claim 1 said whose display is a passing <a thing> 
on scan display. 

Claim 3 The system according to claim 1 said whose display is an interlace format 
display. 

Claim 4 It is the nonlinear system for edit of high definition video data. That random 
access computer reading is possible and the storage system which can be re-written 
in which stores high definition video data per data file, It is high definition video 
system. The high definition video data router which derives reception and this high 
definition video data from said storage system for high definition video data to the 
1st and 2nd outputs, Resizer which has the output at least whose reference standard 
have connected with the 1st output of said router and gives resolution video data 
based on said high definition video data, ****** and high definition video system 
Nonlinear system equipped with the display on which at least said reference standard 
receives resolution video data from the output of said resizer. 

Claim 5 The system according to claim 2 said whose display is a passing <a thing> 
on scan display. 

Claim 6 The system according to claim 2 said whose display is an interlace format 



display. 

Claim 7 Furthermore, the nonlinear system containing the high definition coder / 
decoder connected between said storage systems and said high definition routers 
according to claim 4. 

Claim 8 Furthermore, the nonlinear system containing the digital video effect 
module which accelerates the rendering actuation which said nonlinear system linked 
to said high definition router performs according to claim 4. 

Claim 9 Further Processor with which it is at least two the high definition coder/ 
decoder processors which have been connected between said storage systems and 
said high definition routers, and said high definition video router receives the high 
definition video data of at least two streams from said storage system Nonlinear 
system containing at least one real-time video effect module linked to said video 
data router according to claim 4. 

Claim 10 Further At least one three-dimensions digital video effect module linked to 
said high definition router, They are at least three the coder / decoder processors 
which have been connected between said storage system and said high definition 
video data router. The processor with which said video data router receives the high 
definition video data of at least three streams from said storage system, Nonlinear 
system according to claim 4 which contains at least one digital video effect module 
for every stream connected between said coder / decoder processor, and said high 
definition video data router. 

Claim 11 Said at least one three-dimensions digital video effect module At least at 
least two high definition-reference standards are resizer. At least at least two 
reference standards are - high definition resizer. Nonlinear system according to claim 
10 by which at least a reference standard contains a three-dimensions digital video 
effect module. 

Claim 12 It is the nonlinear system for edit of high definition video data. That 
random access computer reading is possible and the storage system which can be 
re-written in which stores high definition video data per data file, It is high definition 
video system. The high definition video data router which derives reception and this 
high definition video data from said storage system for high definition video data to 
the 1st and 2nd outputs, Resizer which has the output at least whose reference 
standard have connected with the 1st ******** 0 f said router and gives resolution 
video data based on said high definition video data, The high definition output 
module linked to the 2nd output of said router, ****** high definition video system 
At least a reference standard is a video edit system. At least the reference standard 
which has the input which receives the output of said resizer Digital video effect 
module, Nonlinear system by which at least the reference standard with which at 
least the reference standard which visualizes in advance the video data which has 
the additional effect to which at least said reference standard is outputted from a 
digital video effect module contains a monitor is equipped with a video edit system. 
Claim 13 The nonlinear system according to claim 12 which connects said high 
definition output module to a high definition television resolution monitor, and 
performs the check by looking in the degree of maximal solution image of said edit 
video data. 

Claim 14 The nonlinear system according to claim 12 said whose high definition 
video data router is a multi-format router. 

Claim 15 The nonlinear system according to claim 12 by which said router receives 
two data streams. 

Claim 16 Furthermore, the nonlinear system containing at least two resizers 
according to claim 15. 

Claim 17 Furthermore, the nonlinear system containing the digital video effect 
device which receives the output of said resizer according to claim 12. 




Claim 18 The nonlinear system according to claim 12 by which a digital video effect 
module receives the output of said resizer at the rate of said high definition video 
data at least in said reference standard. 

Claim 19 The nonlinear system according to claim 12 by which at least the reference 
standard with which a video edit system receives the output of said resizer at least in 
said reference standard contains a buffer. 

Claim 20 The nonlinear system according to claim 12 by which said high definition 
video system contains a high definition coder / decoder processor. 
Claim 21 It is the approach at least a reference standard edits high definition video 
data using video equipment. The step which receives high definition video data, The 
step which resizes said high definition video data and aligns at least said reference 
standard with the band of video equipment, Step from which at least said reference 
standard adds an effect to said resized high definition and video data by real-time 
using video equipment At least a reference standard on video equipment step which 
visualizes said resized high definition video data containing said added effect in 
advance by real-time the step which carries out the rendering of the high definition 
video data with the degree of maximal solution image with said added effect — since 
— the approach of changing. 

Claim 22 Said step to resize is . The original copy of said high definition video data 
is saved to a data file Approach including performing LISA IJINGU to the copy of said 
high definition video data according to claim 21. 

Claim 23 Furthermore, an approach including storing in a data file the result of said 
step which carries out a rendering according to claim 21. 

Claim 24 It is the resizer which reformats high definition video data and is doubled 
with a bandwidth limit of a low grace system. The input which is the demultiplexer 
which separates the component of high definition video data, and receives high 
definition video data, Demultiplexer which has the output which sends the 
component which said high definition video data separated In order to reformat said 
high definition video data with the input which receives the output of said 
demultiplexer The 1st register set which has the output which sends data at the rate 
determined with a resizing multiplier, The input which receives the output of said 1st 
register set at the rate determined with said resizing multiplier, The 2nd register set 
which has the output which sends data with said high definition data rate, The input 
which receives the output of said 2nd register set with high definition data rate, The 
1st buffer which has the output which sends data with a low resolution data rate It 
has the 2nd buffer which has the input which receives the output of said data from 
said 1st buffer with said low resolution data rate. Resizer which outputs the resized 
video data. 

Claim 25 Resizer according to claim 24 whose components of said high definition 
video data are RUMA and a chroma component. 

Claim 26 Further Resizer which operates on the chroma component of said high 
definition video data Resizer containing the resizer which operates on the RUMA 
component of said high definition video data according to claim 24. 
Claim 27 Resizer according to claim 24 said whose 1st and 2nd buffers are first in 
first out buffers. 

Claim 28 the approach of resizing high definition video data to a low grace video 
data format -- it is -- Step which receives high definition video data Step which 
carries out the filter of the horizontal component of said high definition data at the 
rate of said high definition video data Step which carries out the filter of the vertical 
component of said high definition data at the rate of low resolution data the step 
which outputs the resized video data since — the approach of changing. 
Claim 29 Further Approach containing the step which divides said high definition 
video data into RUMA and a chroma component according to claim 28. 



Detailed Description of the Invention 
0001 

(Field of invention) 

This invention relates to edit of high definition video data. 
(Background of invention) 

It is used, in order that a television (SDTV) resolution editor may exist and at least 
current and a reference standard may edit video data. It is possible to create a 
composite signal (composite) to real-time, and to edit an effect. However, if it is not 
after in many cases storing it in a storage system whenever HD video data which 
may be impossible for a (high definition HD) edit device in cost, and is edited edits, 
and creating an effect in order to edit the video data of high definition television 
(HDTV) resolution, the result of having edited HD video data cannot be checked by 
looking. When more edits are required, HD video data is sent and edited into an 
editor, and it saves to a storage system again. In this way, the edited form can be 
checked by looking. This process requires costs and time amount. 
0002 

By the uncertainty over development of a high definition television (HDTV) resolution 
system and a future criterion, editing HDTV video data in cost efficiency is called for. 
The present SDTV system cannot perform original edit of HDTV compression video 
data or digital video actuation, and cannot acquire the capacity which displays the 
edited HDTV video data with the degree of maximal solution image. 
(Outline of invention) 

This invention offers the system for the real-time rendering of HD video data 
containing the effect which the effect added to high definition (HD) video data added 
which added and real-time beforehand visualized . This system reformats HD 
video data and contains the resizer (resizer) to which it is made at least for a 
reference standard to be settled in the band limit of the (SD) device. 0003 
Therefore, one mode is a system into which high definition video data is edited. The 
storage system in which re-writing is that random access computer reading is 
possible and possible stores high definition video data per data file. Including a 
high definition video data router, high definition video data is sent out to reception 
from a storage system, and a high definition video system sends out high definition 
video data to the 1st and 2nd outputs. Resizer is connected to the 1st output of a 
router. Resizer has the output at least whose reference standard gives resolution 
video data based on high definition video data. The high definition output module is 
connected to the 2nd output of a router. A video edit system includes the display for 
visualizing in advance the video data to which at least the reference standard which 
has the input from which at least a reference standard receives the output of resizer 
added the digital video effect module and an effect like as a reference standard a 
monitor or a computer screen. Video data is a high definition frame rate, and at least 
a reference standard is outputted from a digital video effect module. 
0004 

Another mode is the approach at least a reference standard edits high definition 
video data using video equipment. It resizes so that at least reception and a 
reference standard may set high definition video data by the bandwidth of video 
equipment (resize). An effect is added to the high definition video data which resized 
at least the reference standard using video equipment by real-time, and at least a 
reference standard visualizes in advance the resizing version high definition video 
data containing the effect which carried out whether it would be capable on video 
equipment. The rendering of the high definition video data of the degree of maximal 
solution image containing the added effect is carried out. 



0005 

Another mode is use of the high definition television resolution monitor for checking 
by looking the data edited by the high definition frame rate with the degree of 
maximal solution image. 

Another mode is use of the multi-format router as a high definition video data router. 
0006 

Another mode is resizing high definition video data, holding save the original copy 
of high definition video data to a data file, and the original high definition video 
data in memory, while it has been eternal. 
0007 

Another mode is storing in a data file the result of having carried out the rendering of 
the high definition video data of the degree of maximal solution image to which the 
effect's was added. 
(Detailed explanation) 

Drawing 1 a is the block diagram of 1 operation gestalt of the system into which high 
definition television (HDTV) resolution video data is edited by the single video data 
stream which should be edited. Drawing 1 a contains the (high definition HD) video 
system 104 connected to HD storage system 102 by the bus 144. As for HD video 
system 104, at least the reference standard is connected also to the television 
(SDTV) resolution frame buffer 126 by the bus 145. On the other hand, as for the 
television (SDTV) resolution frame buffer 126, at least the reference standard is 
connected to the display 138. A display 138 can be considered as passing <a thing> 
on or the interlace format like as a computer display or a reference standard 
the (SD) monitor. The data to display relate to a format of HD video data (for 
example, when HD data are a passing <a thing> on format, SD display turns into a 
passing <a thing> on format display). HDTV video input / output module 140 is 
connected to the output of a router 120, and an output is given to the HDTV monitor 
142. Another edit system can find out in the United States patent application entitled 
"EDITING SYSTEM WITH ROUTER FOR CONNECTION TO HDTV CIRCUITRY" by 
Morton Tarr et al. for which it applied on April 2, 1998. 
0008 

What is necessary is just to usually compress a HDTV video signal using about 5:1 
ratio, in order to store a digital HDTV video signal in the transmission band width of 
face of a standard system action (for example, system based on the U.S. television- 
systems committee (NTSC) or a sequential color, and memory (SECAM) 
specification), for example, the NTSC standard makes requirements the aspect ratio 
(ratio of piece pair height) of 4:3, the 525 scanning lines per screen, and a 6MHz 
television signal band. SECAM specification specifies the 625 scanning lines per 
screen, and the television signal bandwidth of 8MHz. A SDTV resolution video format 
is usually the 704-pixel pair of 480 lines, or the 640-pixel pair of 480 lines. By 
contrast, HDTV resolution video data usually makes bandwidth of 30MHz 
requirements, it is twice the number of scanning lines in one frame of this, and it 
changes an aspect ratio into 16:9. Although HDTV resolution video is usually the 
1920-pixel pair of 1080 lines, in passing <a thing> on or an interlace format, 
another format like the 1280-pixel pair of 720 lines is sufficient. HD video data can 
contain all data with resolution higher than SD video data like the data of resolution 
higher than the 525 scanning line which has a component rate higher than 30 frames 
per second in 8 or 10-bit precision. HD data can be interlaced, or can be scanned to 
passing <a thing> on, and this invention is limited to neither of the formats. 
0009 

In 1 operation gestalt of this invention, by operating by real-time and carrying out 
the sequence of the nonlinear high definition storage system 102 at random, the 
single stream edit system of drawing 1 a can perform cut edit, and can output data 



in the sequence shown by the desired video sequence. The storage system 102 is 

good for random to consider as an accessible disk array. 

OOIO 

HD video system 104 of drawing 1 a contains HD video data router 120 which sends 
video data for video data to reception, HD video input / output module 140, a frame 
buffer 122, or resizer 124. Resizer 125 adjusts data with higher resolution to the 
format with low resolution. For example, resizer 124 reformats HD video data, and 
gives the output suitable for the bandwidth of a SDTV device. A SDTV device can 
process the data which usually have the resolution of the less than 525 scanning 
lines which has the component rate of less than 30 frames per second. The output of 
resizer 124 is sent to the SDTV frame buffer 126. Resizer 124 reformats HDTV-video 
data and a SDTV expression enables it to display it on a SDTV device like an NTSC 
monitor or the RGB computer screen 138. Before resizer 124 reformats data, a low- 
pass filter is covered over data and ray ASHINGU of data is avoided. Subsequently, 
resizer 124 generates only desired output data. 
0011 

Drawing 1 b shows an example of resizer. The resizer shown in drawing 1 b operates 
for the RUMA component of HD video data. It is also possible to resize the chroma 
component of HD video data using the same resizer circuit. A demultiplexer 190 
receives HD video data as an input. A demultiplexer 190 divides 148.5MHz HD video 
stream into two 74.25MHz streams. In another side, one of them contains a chroma 
component including a RUMA component, the case where a data rate is 74.25MHz — 
the inside of resizer — setting — a site -- a programmable gate array (FPGA) is used. 
This rate is within the limits of an FPGA device. It is also possible to use a specified 
use integrated circuit (ASIC) with an alternative implementation gestatt. 
0012 

The operation gestalt shown in drawing 1 b performs 5:1 resizing actuation using 5- 
tap filter. However, the number of filter taps can be changed in case other resizing 
actuation is performed. Resizer changes a data rate by 1920/720 of ratios, and 
generates the video based on SDTV. Resizer can expand and contract video in the 
size of all arbitration by changing the value stored in the perpendicular counter 196, 
the level counter 197, and the multiplier storage section 198. 
0013 

The RUMA component of video is inputted into the 1st register set 192 at HD video 
rate, and is inputted into the 2nd register set 193 at the rate subsequently 
determined with a level resizing multiplier. A desired level pixel value carries out the 
multiplication of the five present multipliers to each input pixel, and computes them 
by totaling a result. A multiplier expresses the function to determine the weight 
given to each input pixel. After computing each of a new output pixel, when only the 
value determined by the level counter advances input data, the data suitable for 
computing the following pixel are expressed. In the actuation which reduces an input 
image by the ratio of N:l, a level resizing circuit is an average and generates a new 
pixel every 74.25 clock cycles of IM individual. 
0014 

The output of level resizer is 74.25MHz HD component clock rate, and can be written 
in on FIF0195 on the selected clock cycle. Data are read from FIF0195 at SD rate of 
27MHz, and perpendicular resizing actuation is performed by SD clock rate. Although 
perpendicular resizer operates like level resizer, before it computes the following 
output value, it stores not a pixel but resized Rhine. 
0015 

It is also possible for it to be made to perform the data in a demand clock rate to 
drawing 1 easily using the middle storage element which is not shown. It stores in a 
device or a multiplier and counted value can also be loaded with a host computer or 



other control devices. 
0016 

In order to perform authoring, processing, and a display of multimedia data, there 
are seven kinds of usable systems (for example, HD video system 104). These 
systems change data, define the combination of different data, and create new data, 
and they can be used for them in order to display data on a user. The various 
techniques for realizing these kinds of systems are well-known in this technical field. 
0017 

Multimedia authoring, processing, and a regeneration system usually have the DS 
showing a multimedia presentation. This DS means ultimately the clip of digitization 
video or a source material like an audio, and the time amount range in the source 
material which probably defines a ****** clip using the identifier of a meaning, or 
the identifier of a source material like a file name. An identifier can be made into the 
format of being used with an equivalent data file list in order to identify the file name 
of a source material. It is also possible to convert the time amount range in the 
source into the byte range in a corresponding file using an index. This byte range can 
be used with the segment table of a file, and can identify the segment of required 
data, and the storage which detects data. 
0018 

A part of multimedia presentation can be expressed using the list structure. A clip 
can include the range in the reference to a source identifier, and the list concerned. 
Generally, there is such a list for every media truck within a time presentation 
(temporal composition). The DS which can be used since a presentation is expressed 
is various. In addition to the list structure, still more complicated structure is shown 
in the PCT public presentation application WO 93/21636 exhibited on October 23, 
1993. The example of representation of others of a multimedia presentation Avid 
Technology, Open Media Framework Interchange Specification from Inc, Advanced 
Authoring Format from Multimedia Task Force, What this is also defined by by Bento 
from Apple Computer is included as shown in QuickTime from Apple Computer, 
DirectShow from Microsoft, and the PCT public presentation WO 96/26600. 
0019 

The DS used since it explains previously and a multimedia program is expressed can 
take and display a synchronization using the data of many classes. The most general 
example is a television program containing the motion video (in the case many two 
or more streams or trucks) accompanied by an audio (in the case many four or more 
streams or trucks), or manufacture of a film. 
0020 

since video and audio data can be stored in a different data file and it can combine 
with arbitration — ******** — jf the demand to the data of a data file is managed 
efficiently, a still higher throughput can be acquired. For example, application 
specifies the stream which can read data, and if required, it can judge the amount of 
data which should be read. The process which manages this kind of reading 
actuation is shown in U.S. Pat. No. 5,045,940. Generally, which stream judges 
application about whether it has data of the minimal dose obtained for a display. 
There is sufficient amount of memory for the data which should be reproduced to the 
stream, and when an certain amount of data can be read efficiently, the data is read 
from a file. When it judges with the data to a certain stream being required, each 
segment of data is required from one store chosen from the store with which the 
segment is stored. In order to identify the file demanded from a store, an edit 
system can convert the DS showing a presentation into the range in a file name and 
its file. 
0021 

In drawing 1 a, disk buffer memory 114 can be used for example, as a round buffer, 



and receives a series of digital still images. Each still image can express the single 
frame of the motion video data from HD storage system 102, i.e., the 2 field, or can 
express the single field. This is processed in order to carry out the rendering of the 
edited video data. For example, application software like the software used in the 
Media Composer edit system by Avid Technology and Inc. can reproduce and operate 
a series of digital still images, and can put them in in buffer memory 114. The disk 
buffer memory 114 holds many video frames, sends these out to a router 120 
through one or more a coder / decoder processors 116 (KODEKKU), and reduces the 
overhead in the case of using a linear device with the nonlinear access storage 
system 102. 
0022 

A display 138 like a SDTV monitor or a computer display is connected to the SDTV 
frame buffer 126, and it is used in order to visualize in advance the HDTV composite 
signal processed so that edit specified by a user might be included. The HDTV 
monitor 142 is used in order to check by looking the HDTV composite signal including 
the edit which visualized the output in advance on reception and a SDTV device from 
HDTV video I/O module 140 which carried out the rendering with the degree of 
maximal solution image. 
0023 

It is also possible to control the data flow between the components in drawing 1 a, 
drawing 1 c or drawing 3 and drawing 5 thru/or drawing 7 using the hardware data 
flow interface which makes it possible to interconnect an asynchronous data- 
processing component using the interconnect protocol which controls the data flow 
between processing components. Such an interface The United States patent 
application number 08th of June 20, 1997 application / No. 879,981, "APPARATUS 
AND METHOD FOR CONTROLLING TRANSFER OF DATA BETWEEN AND PROCESSING 
by CRAIG R.FRINK of April 3, 1998 application The United States patent entitled OF 
DATA BY INTERCONNECTED DATA PROCESSING ELEMENTS", Craig R. of April 3, 
1998 application "A PACKET by FRINK et al. PROTOCOL FOR ENCODING The patent 
application entitled AND DECODING VIDEO DATA AND DATAFLOW SIGNALS AND 
DEVICES FOR IMPLEMENTING THE PACKET PROTOCOL", And "METHOD by Craig 
R.Frink It is indicated by the United States patent application entitled AND 
APPARATUS FOR CONTROLLING DATA FLOW BETWEEN DEVICES CONNECTED BY A 
MEMORY." ** shall be contained also in this application by this reference among 
these. By the flow control, a processing component can be made independent of a 
medium or a format. This system is not limited to a flow control, but other interfaces 
and synchronization approaches are also usable. For example, in drawing 1 a, it is 
also possible to control the data flow between the data between the storage system 
102 and HD video system 104, HD video system 104, and the SDTV frame buffer 
126 using a data flow interface. 
0024 

As the system shown in drawing 1 a is changed and it is shown in drawing 1 c, you 
may make it include external HDTV KODEKKU 160 which has HD-D5 KODEKKU of 
Panasonic for the HDTV video store using a tape. It can use for the compression and 
expanding of HDTV video which are too large for format compatibility with the 
PanasonicHDTV record approach being acquired, and storing within a limit of an edit 
system by using this KODEKKU in the non-linear-editing (NLE) application shown in 
drawing 1 a. Panasonic HDTV KODEKKU can be permuted by HD KODEKKU 116, and 
is connected to the disk buffer memory 114 and the HDTV router 120 using the 
conventional approach (interface 259M and the 292M grade of Society of Motion 
Picture and Television Engineers (SMPTE) specification). This connection method is 
similarly applied to other devices containing a video data router, video I/O, and other 
KODEKKU devices (Sony HDCam, MPEG 2 HL@PP, etc.). 



0025 

When using the digital video effects (DVE) (decomposition, a superimposition, 
positioning of a key image, etc.) accompanied by many streams, a lot of rendering 
time amount may be used for the real-time single stream HDTV edit system shown 
in drawing 1 c. Although DVE added using the HDTV edit system of drawing 1 c can 
be checked by looking once it carries out the rendering of these, on the whole, it 
cannot be checked by looking at the time of playback. 

W?th 6 l operation gestalt, rendering time amount can be shortened by adding the 
hardware DVE module 150 as shown in drawing 1 d, and changing the system of 
drawing 1 a. It connects with the HDTV video data router 120, for example, the 
hardware DVE module 150 is 3D. It is used in order to accelerate video actuation 
actuation like DVE or 2D LISA IJINGU. With this operation gestalt, including a 
hardware DVE module, and a frame buffer 122 and the 2nd same frame buffer, in 
addition to a video channel, an edit system stores an alpha channel, and is equipped 
with the ****** bandwidth for the concurrent access of two streams (namely, one 
writing in one read-out). Although this system operates like the system of drawing 1 
a instead of using software and reading the contents of the frame buffer, when 
carrying out the rendering of the DVE, a system reproduces one video stream, on the 
other hand (for example, 122), a frame buffer inputs it, subsequently it reproduces 
the 2nd stream, and inputs it into the direct DVE module 150. Reading the v.deo 
stored in the frame buffer (for example, 122) to compensate for the video recovery 
from a disk (for example, 108), both streams of both pass a DVE module. It stores in 
the 1st frame buffer of addition of the obtained video, and the obtained video turns 
into source video of consecutive DVE, or can be again stored in a disk. 

?n°another modification of drawing 1 a shown in drawing 1 e, the addition of the 2nd 
video channel from the storage system 102 enables it to reproduce many video 
streams to coincidence for real-time DVE , and it becomes possible to visualize DVE 
actuation by real-time further. This includes adding the additional disk buffer 114, 
additional HD KODEKKU 116, and HD-1080 additional frame buffer 122 to the 
system of drawing 1 a. In order to carry out the rendering of many video streams, 
the frame buffer of a twice as many HDTV data rate as this and resolution is used for 
two duplex stream systems, and they capture video and an alpha channel (if an 
alpha channel is storable in the video which carries out a rendering, the rendering 
only of the background will not be carried out but it will become possible to 
compound a complete view). Moreover, a frame buffer is equipped also with the 
device for the frame delay compensation at the time of changing I/O of an element 
to the video pipeline who connects a video device within a system. 

0028 ,.. 

using the HDTV video device shown in drawing 1 a as a separate structure (the case 
where PanasonicHD-D5 KODEKKU is used - like), or putting it on the exterior of a 
computer **** - (-- general, when using conventional linear video equipment - as 
--) - or you may also include in a single design (common in the NLE device using 
almost all computers ~ as). 
0029 

With 1 operation gestalt, when the number of the streams included in a composite 
signal exceeds the real-time capacity (the above-mentioned system two streams) of 
a system including the 2nd video channel from the storage system 102, a non-real- 
time hardware auxiliary rendering can be used for the just explained modification 
system of drawing 1 a. For example, in the system shown in drawing 1 f, the first 
two streams can be combined among three-layer composite signals (three 
coincidence video streams), and the result can be stored in a frame buffer. Many 



frames within a sequence can be processed for effectiveness. The 3rd frame is added 
to the intermediate result stored in the HDTV frame buffer (for example, 114) by 
using only one side of the video stream channel from a storage system (for example, 
102). A video (and alpha) channel can also be stored in a disk in the state of a 
compression condition or un-compressing. Moreover, with the alpha channel relevant 
to it for video, when one side is generated, this system can be stored and can be 
combined with a complete view stream. Moreover, it is also possible to reduce the 
overhead which stores a background and stores an alpha stream in a disk. 
0030 

The system changed as mentioned above is equipped with real-time processing 
capacity, and improves rendering time amount. However, since the number of 
KODEKKU used within a system serves as a limit of real-time actuation, a non-real- 
time rendering is used in many cases. The addition of the 3rd video channel enables 
it to reproduce these, recording one side of two video streams. In case more video 
layers (equal to many streams) are combined, by real-time, a middle composite 
signal can be stored in a disk and can be applied immediately. It is also possible to 
store both video and a related alpha stream (for it to have compressed using run 
length encoding) for simplification, or to store separately. It is also possible to 
capture video using the store using a tape, whenever it creates an effect. In addition, 
by the additional video channel (and HD KODEKKU), video can be stored in a disk by 
real-time, and can be used for the continuing composite signal. 
0031 

Drawing 2 is the block diagram of the SDTV edit system used for the real-time HDTV 
edit and the digital visual effect prior visualization by 1 operation gestalt of this 
invention. This system enables edit of low cost by visualizing a HDTV video effect in 
advance using an available SDTV device, and it makes it possible to edit a video 
sequence further, saving the video which stored video in the storage system 202 in 
the original HDTV formats (for example, un-compressing, HD-D5, HDCam, MPEG 2 
HL@PP, etc.). In addition, the system shown in drawing 2 is compatible with a fully- 
equipped real-time HDTV system. 
0032 

As for the system shown in drawing 2 , at least the video data storage system 202, 
HD video system 204, and a reference standard contain the television non-linear- 
editing system (SDTV NLE) 206. The storage system 202 stores HDTV video in digital 
one using compression and a non-compressed format including a nonlinear store like 
a disk 208. The video stored on a disk 208 is not changed in a non-linear-editing 
process, but the generating loss (generation loss) when using compression is 
avoided. Therefore, when operating the stored video frame, the original frame 
remains in the untouched state and creates a new frame or a frame sequence for 
new video. The storage system 202 can reproduce many streams of the HDTV video 
of the degree of maximal solution image stored in compression or a non-compressed 
format, and the maximum data rate, and can generate real-time transition effects 
(for example, wipe, decomposition, a picture Inn picture, etc.). Moreover, the 
storage system 202 can be fluctuated according to the number of streams in an edit 
system including the disk data router 210 and a disc controller 212. 
0033 

HD video system 204 is connected between HD storage system 202 and the non- 
linear-editing system 206. The video system 204 takes the synchronization of the 
video data delivered between the storage system 202 and the HD router 220 through 
HD KODEKKU 216 including the HD disk buffer 214. HD video data compressed 
before for storing is elongated using HD KODEKKU 216, and it sends to the non- 
linear-editing system 206 after that. Moreover, it is also possible to use HD 
KODEKKU 216, to compress or elongate, and to store HD video data. The HDTV 



video data router 220 judges whether it sends of the output to the non-linear- 
editing system 206, HD storage system 202, or HDTV video I/O240 for the 

ability taking in HD video data. 
0034 

This is formatted and the requirements for bandwidth of a repair non-linear-editing 
system are made to suit by incorporating HD video data from HD video data router 
220, and adjusting the space resolution of HDTV video by video filtering and re- 
sampling using resizer 224. Resizer 224 changes HD video data into SDTV resolution 
by real-time. This enables it to display the SDTV expression of a HDTV image on a 
standard NTSC monitor or a RGB computer screen. HDTV video can hold the original 
aspect ratio (16:9 or 4:3), and a SDTV system can operate using passing <a thing> 
on or an interlace format in these aspect ratios. 
0035 

Drawing 2 contains the SDTV frame buffer 226 in the HD video 204 and a SDTV 
video effect inter module. It is also possible to control the process filled up with the 
SDTV frame buffer 226 using the hardware which raises system response nature and 
makes low the latency by the SDTV frame buffer 226. The HDTV video system 204 
generates SDTV data at a fixed rate, and, on the other hand, can make a SDTV 
pipeline's data rate adjustable (setting to a data flow system). However, the average 
data rate of fixed flow data is about 21MHz. A SDTV pipeline manages the reception 
of the data from a frame buffer 222, using an effective signal as a processing 
pipeline and a part of interconnect. Software like the software used in the Media 
Composer edit system by Avid Technology and Inc is possible also for managing a 
frame buffer 222 by use of round Rink Liszt direct memory access (DMA) structure. 
Since the data rate of the video which reaches a frame buffer 222 is fixed, after a 
video flow begins, in the Media Composer edit system by Avid Technology and Inc, 
software like **** software can pursue the condition of a SDTV frame buffer pointer, 
can avoid the overrunning (overrunning) of a buffer, and can avoid read-out of a 
buffer when data are not yet effective. 
0036 

The non-linear-editing system 206 is used for real-time prior visualization of edited 
HD video data. The SDTV digital video effect module 232 adds desired transition 
effects (for example, wipe, decomposition, a picture Inn picture, etc.) to digital video 
data, or operates digital video data (for example, Paige Carl, a ripple, etc.). The 
video data containing an effect is blended with the SDTV effect module 232, and 
outputs the edited video data to SDTV video input / output module 234. The video 
data which has an effect can be visualized in advance on the SDTV monitor 236 or 
the computer display 238. Prior visualization includes a real-time check by looking of 
under the degree of maximal solution image of the effect applied to the video data. 
Prior visualization can save time amount. Whenever it saves the video data which 
could check by looking, adding an effect to video data and was edited to a tape or a 
disk or desires a new effect, it is because need to search the new part of this video 
data or video data and it is not necessary to edit it. An effect can be added using SD 
device, and since HD device is unnecessary to prior visualization, the cost of edit is 
reducible. 
0037 

Drawing 3 shows the alternative implementation gestalt of the SDTV edit system of 
drawing 2 . In drawing 3 , a SDTV frame buffer (for example, 226 of drawing 2 ) is 
not used, but it is SDTV. The NLE system 306 is connected to the direct resizer 324. 
The HDTV frame buffer 322 and the HDTV video router 320 control the flow of data 
by this operation gestalt. It is also possible to use an interface separate for read-out 
of memory and writing in this system. Three alternate methods which can be used in 
order to carry out the rendering of the HDTV video data to the degree of maximal 



solution image using the system of drawing 3 are explained below. 
0038 

In one of the rendering approaches using the system shown in drawing 3 , the SDTV 
prior visualization system 206 carries out the rendering of the HDTV video data. The 
HD frame buffer 322 captures the HDTV video searched from the storage system 
302, and edit of an effect is processed by the non-real-time one. A frame buffer 322 
stores two or more HDTV frames, and helps capture of a frame, and storing on a 
disk. 
0039 

At the option using the system shown in drawing 3 , the rendering in the degree of 
maximal solution image of HDTV video can also be performed by using a frame 
buffer 322 and a real-time HDTV video device. It is also possible to carry out the 
rendering of the non-compressed frame of much HDTV videos for a non-compressed 
middle composite signal, using a frame buffer 322 as the temporary storage section. 
This elongates a frame, creates an effect and becomes possible by adding an 
additional layer combining non-compressed HD video which stored the intermediate 
result in the frame buffer 322, and subsequently stored it. The limit to the number of 
the intermediate frames in the single composite signal which carries out a rendering 
is decided by memory size of a frame buffer 322. 
0040 

In the option using the system of drawing 3 , although the rendering in the degree of 
maximal solution image of HDTV video can process the video data of the degree of 
maximal solution image, it may include using the non-real-time hardware which is 
not real-time. A frame buffer cuts a real-time instant HDTV data rate from the low 
data rate of a digital video effect. 
0041 

Actuation of drawing 2 is explained from this, relating with the flow chart of drawing 
4 . In step 405, the HDTV video data router 220 captures HDTV video data, and 
sends the HDTV video data which should be stored in step 410 to the disk 208 in HD 
storage system. HD video data to store may be compressed or may not be 
compressed. HDTV video data can also be captured also capturing by real-time 
from the video sources (for example, a videocassette recorder, a camera, 
etc.)/ and by the approaches (for example, a local area network (LAN), a digital 
linear tape (DLT), etc.) using a computer. In step 415, the HDTV video data router 
220 searches video data from HD storage system 202. The video data router 220 can 
be searched for edit of video data of one frame, or a nonlinear frame sequence. 
Before HDTV video data is sent for storing or retrieval, it may be compressed or 
elongated by HD KODEKKU 216. HD disk buffer memory 214 controls the flow of 
video data by exchanging flows-of-control information for a disc controller 212. It is 
also possible to control the flow of data using the interface of others like a 
handshaking protocol or the above-mentioned hardware data flow interface. 
0042 

Based on HD video data, resizer 224 adjusts HDTV video data and is SDTV. The 
output of SDTV resolution video data is given by doubling with the SDTV bandwidth 
of the NLE system 206. As discussed previously, in order to double a digital HDTV 
signal with standard transmission band width of face, the compression ratio of 
abbreviation 5-1/2:1 is needed. 
0043 

The resized HDTV video data is sent to the SDTV frame buffer 226. The SDTV frame 
buffer 226 is used in order to synchronize a transfer of the video data in the SDTV 
edit system 206 in step 425. The SDTV edit system 206 is added to the HDTV video 
data which resized the effect in step 430. An effect may also contain transition 
effects, such as wipe and fade. 



0044 

In step 435, the resized HDTV video data containing the added effect is visualized in 
advance on the SDTV monitor 236, the computer display 238, or other displays. By 
this, a user can check by looking the effect added from the variable with the low 
resolution, and avoids the problem accompanying the generating loss produced by 
carrying out the rendering of the new video for every effect, and compression of 
video data. 
0045 

In step 440, when you need video data further for edit, a process searches return to 
step 415 and a router searches video data from HD storage system 202. When you 
do not need video data any more for edit, in step 445, it carries out the rendering of 
the video data visualized in advance to the non-real-time one with the SDTV effect 
module 232 to the degree of maximal solution image using above-mentioned 
hardware or an above-mentioned software rendering device. 
0046 

In step 450, the video which carried out the rendering can be stored in the storage 
system 202, and, subsequently can be searched with usual on an HD device like the 
HDTV monitor 242 in step 455 for the check by looking with the degree of maximal 
solution image. 
0047 

Actuation of drawing 3 is similar to actuation of drawing 2 explained previously, 
referring to drawing 4 . However, it sets to step 425 and is SDTV. A transfer of 
delivery and video data controls the HDTV video data which resized the NLE system 
(namely, 306) from resizer 324, excluding therefore a SDTV frame buffer (for 
example, 226 of drawing 2 ) by HD-1080 frame buffer 322 to the direct SDTV edit 
system 306. 
0048 

Drawing 5 is the block diagram of the real-time HDTV edit system of the maximum 
equipment. HDTV video is used for it although the system in drawing 5 is equivalent 
to a SDTV edit system. This system is real-time HD for every video channel (color 
amendment, resizing, FUROPPU, an image crop, etc.). The DVE module 554 is 
included and it is HDTV. 3D By using the DVE module 550, it has the 3D digital video 
effectiveness capacity much more complicated for a transition effect and video 
actuation (Paige Carl, a ripple, projection warp, etc.). This system is 3D. The HDTV 
video router 520 which contains a mixer and downstream keying (DSK) hardware 
further can also be used using the DVE module 550, and many video streams can be 
compounded. Moreover, this system is a quiescence graphic (for example, title) 
and Silicon about real-time video. OpenGL by Graphics 3D The DSK hardware 
containing a product like a graphics DVE accelerator combined with the animation 
which carried out the rendering by real-time using software like the graphics 
accelerator 552 which generates the video outlet which has a key is also included. 
HDTV video compression is possible if HD KODEKKU 516 is used, but if non- 
compressed video input / output path 518 is used, non-compressed video recovery is 
also possible. The system shown in drawing 5 contains three video channels, in order 
to carry out a rendering by real-time, reproducing. 
0049 

Drawing 5 operates by real-time using HDTV resolution and a data rate. It is also 
possible for the system of drawing 5 to process SDTV video by real-time using the 
same structure in addition to processing of HDTV video. This system can derive SDTV 
video through the HDTV router 520 (a single and many streams) combining HDTV 
video. DVE, mixing, and other functions adjust HD video data to SDTV processing 
and the data rate of HDTV resolution to low resolution. 
0050 



It is HDTV as shown in drawing 5 a. 3D Modification to drawing 5 can also be added 
to the DVE module 550, and, as a result, it is SDTV. 3D It becomes a hybrid system 
using the DVE module 572, the HD-SD resizer 570, and the SD-HD resizer 574, and 
is HDTV. 3D A DVE module is replaced. When this modification system is used, SDTV 
resolution is used and it is 3D. D VEMO joule can be visualized in advance by real- 
time. 
0051 

Drawing 6 is the block diagram of the real-time non-compressed 3 stream real-time 
HDTV video system which used the acceleration graphics port (AGP) interface for the 
bus protocol to the circumference connection interface (PCI) bus for example, 
between HD video system 604 and the computer memory 652. 
0052 

Using two or more PCI computer buses, for example, the PCI interface of 
FibreChannel, the system of drawing 6 separates a HDTV data stream (64-bit PCI or 
66MHzPCI(s) which are used for real-time HDTV a non-compressed format), and 
brings a video stream together in common high-speed host computer memory. It 
becomes possible to send out disk data through a PCI bus, to transmit to the host 
memory 652, and to transmit to HD video processing system 604 through the high 
performance data interface (AGP) 654 by this, subsequently. A system 604 accesses 
the host memory 652 using many DMA Channels in the AGP interface 654, 
reproduces many video streams, and captures many video streams. AGP A DMA 
device can also be carried out as a separate DMA device or DMA in which many 
coincidence contexts are possible. With 1 operation gestalt, a PCI bus can maintain 
the component bandwidth of 1 or 2 streams of HDTV compression video data, and 
becomes per 30 / about stream thru/or 40 megabytes. AGP maintains the 
bandwidth of non-compressed HDTV video data. 
0053 

Moreover, it is also possible to form connection between the storing system 602 and 
HD video system 604, and to carry out the rendering of the software DVE using the 
AGP interface 654 and the high-speed host memory 652. A video frame buffer which 
is needed for the DVE actuation which needs a rendering, time amount stem 
correction, an input, and/or an output frame buffer is allotted in the host CPU 
memory 652, and HD video system 604 can also reduce the cost of video system 
hardware. 
0054 

The HDTV video stream 604 The disk data buffer 652 (in order to carry out the 
buffer of the data from a disc controller 612), The frame buffer 614 for carrying out 
the buffer of the latency at the time of accessing the disk data buffer 652, HD 
KODEKKU (not shown) (whether it holds inside on a PCI board) For example, the 
interconnect section to the interface, HDTV video 1/0640, and the monitor 642 which 
are held outside using the digital interconnect section like HD-D5 KODEKKU of 
Panasonic is included. When required for effect processing (rendering) of the degree 
of maximal solution image, the degree HDTV frame buffer 622 of maximal solution 
image operates with the maximum data rate, and captures the HDTV frame. Video 
data is transmitted between a computer system and the HD frame buffer 622, and 
resizer 624 carries out the subsample (subsample) of the HDTV video of the degree 
of maximal solution image to the SDTV data rate of the aspect ratio (or 4:3) of 16:9. 
As discussed previously, the resized video is transmitted to a display 638 (namely, 
monitor) and a non-compressed SDTV video edit system (not shown), and real-time 
DVE and edit prior visualization are performed. 
0055 

A data compression is needed when the number of the system which receives 
constraint by PCI and the storing throughput, and non-compressed storage elements 



is impossible in cost. The storage system of 1 operation gestalt of this invention can 
be made independent of a system if the approach by which symmetrical data access 
is obtained exists distributing a video bandwidth range to a HDTV device. When 
compressing HDTV video, or when using non-compressed HDTV video, it is good to 
use many storage controllers and partition PCI buses. 
0056 

The system of drawing 6 a is the block diagram of a real-time 3 stream non- 
compressed HDTV video system. The number of PCI interfaces is fluctuated with a 
bandwidth demand like the number of the separate PCI bus segments corresponding 
to the bandwidth of a system. A separate PCI bus segment is used for the system 
shown in drawing 6 a every HDTV non-compressed disk data buffer 614. The storage 
system 602 is equipped with the share data access approach, and derives a disk data 
packet to the disc controller corresponding to the HDTV stream which derives data. 
The HDTV router 620 can prepare connection of a video stream, when video 
actuation is performed within a device like the DVE module 650. 
0057 

Drawing 7 is the block diagram of a HDTV video system. Each video channel 
interface shown in drawing 7 is carried out in single ASIC. Each ASIC is a 64-bit PCI 
interface (741), a 1500 MB/s memory interface for disk data buffer 714, and HD. The 
DVE module 754 can be included. 2D DVE actuation can contain color amendment, a 
chroma and Le Maquis generating, video LISA IJINGU, and a motion effect. Each 
video channel interface can process video by real-time, and can interface with the 
HDTV data router ASIC using the high-speed desktop video interconnect section. The 
duplex HDTV frame buffer which has a PCI interface is controllable by single ASIC. A 
module 706 can also be used in order to check a HDTV composite signal by looking. 
0058 

It is included that "on-Rhine" is maintainable checking by looking the video which 
was edited and was edited into one of the advantages of this invention, without 
having returned to the disk or the tape and searching some videos. When at least a 
reference standard uses a device and it edits HDTV video, reducing required real- 
time HDTV configuration significant work is included in another advantage. This 
invention can combine the video data of HD and SD format, and can visualize an 
effect in advance effectively by real-time using the device of low resolution. 
However, the image visualized in advance is not the degree of maximal solution 
image. In addition, in order to prevent degradation, the original video data file is 
stored in the store, and edit is performed to the copy of a video data file. 
0059 

The edit system of this invention enables association of HDTV, a SDTV video device, 
and a data rate in the same system using a data flow video pipeline, the interconnect 
section, and the DMA controller combined through FIFO. If locked in a HDTV 
reference signal, whenever an output frame will be obtained from HD subsystem, 
only the SDTV system should synchronize with an output frame. If an external HDTV 
device is used with this invention when creating video from the HDTV source for 
SDTV distribution and, the 27MHz clock reference of a SDTV system will generate the 
74.25MHz clock of a HDTV subsystem and its external configuration object. 
0060 

although some operation gestalten were explained until now, probably, the thing 
which old explanation is mere instantiation and showed not as limitation but as an 
example and which is not boiled too much will be clear to this contractor. Many 
modification and other operation gestalten are this contractor's within the limits, and 
it is considered that they are the things applicable to the range of this invention. 
Brief Description of the Drawings 
Drawing 1 



Drawing 1 a is the block diagram of a high definition television (HDTV) resolution edit 
system. 

Drawing 1 b is drawing showing the resizer by 1 operation gestalt, and is **. 
Drawing 1 c is drawing showing modification to the edit system of drawing 1 a. 
Drawing 1 d is drawing showing modification to the edit system of drawing 1 a. 
Drawing 1 e is drawing showing modification to the edit system of drawing 1 a. 
Drawing 1 f is drawing showing modification to the edit system of drawing 1 a. 
Drawing 2 

At least the reference standard used for HDTV edit and digital visual effect prior 
visualization is the block diagram of a television (SDTV) edit system. 
Drawing 3 

It is the block diagram of the SDTV edit system which interfaces with HDTV resizer 
directly by 1 operation gestalt. 
Drawing 4 

It is the flow chart which indicates how the SDTV edit system of drawing 2 is used 
for HTDV edit according to 1 operation gestalt. 
Drawing 5 

Drawing 5 is the block diagram of all the functions of the real-time HDTV edit system 
by 1 operation gestalt. 

Drawing 5 a is the block diagram of the modification system of drawing 5 . 
Drawing 6 

Drawing 6 is the block diagram of the real-time non-compressed 3 stream HDTV 
video system which used the acceleration graphics port. 
Drawing 6 a is the block diagram of the modification system of drawing 6 . 
Drawing 7 

It is the block diagram of a HDTV video system. 
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Claim 1 It is a nonlinear video edit system, 

That random access computer reading is possible and the storage which defines the 
image which is the storage which can be re-written in, has the scanning line with 
more said video data than 525, and has a component rate higher than 30 frames per 
second which stores video data per data file, 

The nonlinear editor which is a nonlinear editor which defines a video program as a 
sequence of the part of said data file, and defines each part of a data file by the 
range in the reference to said data file, and said data file, 

The means which reads said video data from said storage according to said defined 
video program, 

At least one resizer which gives the output video data which has the input which 
receives said video data read from said storage, has the space resolution of the 
scanning line fewer than 525, and has a component rate lower than 30 frames per 
second, 

At least one video effect module which has the output which gives the input which 
receives the output of said resizer, and the video data edited by real-time according 
to one or more effects defined for the video program, 

The display which has connected with the output of said at least one video effect 
module, and performs prior visualization of said video data edited and resized, 
******** nonlinear video edit system. 

Claim 2 The system according to claim 1 said whose display is a passing <a thing> 
on scan display. 

Claim 3 The system according to claim 1 said whose display is an interlace format 
display. 

Claim 4 Further, 

It is a high definition video system, 

A system equipped with the high definition video system which connected 
said resizer to said 1st output, and connected the high definition display to 
said 2nd output including the high definition video data router which 
derives reception and this high definition video data from said storage 
system for high definition video data to the 1st and 2nd outputs according to 
claim 1. 

Claim 5 The system according to claim 4 said whose high definition video data 
router is a multi-format router. 

Claim 6 The system according to claim 4 by which said high definition video router 
receives two data streams. 

Claim 7 The system according to claim 6 said whose at least one resizer is two or 
more resizers at least. 

Claim 8 The system according to claim 1 by which said video effect module receives 
the output of said resizer at the rate of said high definition video data. 
Claim 9 Furthermore, the system according to claim 1 by which at least the 
reference standard which receives the output of said resizer is equipped with a 
buffer. 

Claim 10 The system according to claim 4 by which said high definition video 
system contains a high definition coder / decoder processor. 

Claim 11 It is the approach at least a reference standard edits high definition video 



data using video equipment, 

It is the step which is the nonlinear editor which defines a video program as a 
sequence of the part of a data file, defines each part of a data file using a nonlinear 
editor by the range in the reference to said data file, and said data file, and contains 
one or more effects, 

The step which receives high definition video data from that random access 
computer reading is possible and the data file stored on which can be written in 
the storage according to said defined video program, 

The step which resizes said high definition video data and aligns at least said 
reference standard with the band of video equipment, 

The step from which at least said reference standard adds an effect to said resized 
high definition and video data by real-time using video equipment, 
The step to which at least a reference standard visualizes said resized high definition 
video data which contains said added effect on video equipment in advance by real- 
time, 

The step which carries out the rendering of the high definition video data with the 
degree of maximal solution image with said added effect, 
since — the approach of changing. 
Claim 12 Said step to resize is , 

The original copy of said high definition video data is saved to a data file, 
LISA IJINGU is performed to the copy of said high definition video data, 
The approach of ****** claim 11 publication. 

Claim 13 Furthermore, an approach including storing in a data file the result of said 
step which carries out a rendering according to claim 11. 
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